Assessing the Ability of Matching to Address Attrition Bias in a Randomized Experiment using Data from the Rand Health Insurance Experiment

It is well known that non-random attrition can lead to bias in estimating treatment effects from a social experiment that is based on random assignment. If the randomized intervention suffers from non-random attrition, the intent to treat (ITT) estimator is biased and the IV estimator of the treatment effect is also inconsistent (Frangakis and Rubin 1999). DiNardo, McCrary and Sanbonmatsu (2006) propose an approach to correct attrition bias, but their approach requires an additional randomization. In principle, propensity score matching can eliminate or mitigate the bias due to nonrandom attrition. This study investigates how well matching achieves this goal after we introduce a plausible form of non-random attrition in the well-known Rand Health Insurance Experiment (RHIE). Specifically, we introduce attrition in a less generous insurance plan sample at the end of year 1 of the experiment on the basis of high health care expenditures in year 2. We then assess whether matching can eliminate or mitigate the bias in estimating the treatment effect of switching plans by comparing our matching estimates to the experimental results. Since the data on pre-experimental health care expenditures is selfreported and unreliable, we use the individuals’ year 1 expenditure rank within their respective plans as one of our conditioning variables. While this is a post-experiment (treatment) variable, its use will be valid if an individual’s health expenditure ranking within a plan is not affected by health insurance plan assignment, i.e. the individuals are rank-order stable across the plans. We find that when we use year 1 expenditure rank as a conditioning variable and the outcome variable is defined as chronic condition expenditures, matching eliminates about half of the attrition bias. However, this is not the case when the outcome variable is defined as the total health care expenditures. Finally, without conditioning on year 1 rank, matching cannot mitigate attrition bias for either outcome variable.